For military-use or security-use systems, multiple competing requirements complicate sensor design, especially infrared (IR) sensor design. Quantitatively, an ideal military/security system needs a fast sensing mode that operates at maximum rates of 1500 to 5000 frames per second, combined with a slow, staring capability that can match the sensitivity of a high-quality 30 frame per second sensor. It needs resolution suitable for fine-grained images, and a dynamic range (the difference between the darkest signal that can be measured and the brightest signal that can be measured) of over one million times (10 million or more is optimal). It should provide new types of raw data that enable qualitative improvements in anti-clutter techniques, that is, techniques to distinguish between threatening events and other manmade events of no interest (non-threatening events).
No detector technology exists that meets these goals. Systems are typically customized to support a specific requirement; for example, they are able to function in an optically bright tactical military environment or an optically dark environment, but not both. Or, as another example, they are optimized for speed by sacrificing resolution, or optimized for resolution by sacrificing speed.
Design approaches employed today focus primarily on enhanced resolution, i.e. more pixels and smaller pixels. This is important, but it only addresses one aspect of the multiple critical needs outlined above. A second area of design emphasizes “all digital” sensor chips wherein signal digitization is performed on the sensor readout circuit instead of off the sensor chip assembly. Some of these designs have the potential to address the need for larger dynamic range.
When higher data processing speeds are required, either a design tradeoff is made—fewer pixels for less resolution and therefore less data to be processed—or output channels are added to enable the higher speed. For example, rather than using two or four pixel data signal lines, eight or sixteen could be used. This achieves a 2× to 8× speed improvement, but adds complexity to the external chip control electronics and to the camera mechanical housing, which is typically a cryogenic dewar flask. An improvement of 25× to 50× is what is really required (for example, from 60 frames per second to 2000 frames per second or greater).
Current IR sensors achieve a dynamic range of less than 20 thousand, and this often presents a problem in real applications. In comparison, the human eye can see a total brightness range of over one million times. In visible photography, systems have been produced that achieve a dynamic range of over one million—these are called HDR for High Dynamic Range. Also, so-called sigma-delta digital designs, if implemented with 20 or more encoding bits, have the potential for a dynamic range of one million. However, both HDR and the sigma-delta approach result in performance tradeoffs. A different approach is used here.
Clutter rejection methods have employed two-color radiometric techniques, which are highly effective, but clutter in the IR bands remains a difficult problem. Two-color methods typically compare the brightness in two IR bands to determine the approximate temperature range of an object or event. The temperature information is then used to determine whether an observed event is threatening, such as the launch of a shoulder-fired missile, or benign, such as the glint of the sun off a car windshield. However, these techniques are limited by how precisely the temperature is determined. They are also limited because some benign events have similar temperatures to threatening ones. For example, it would be difficult to distinguish a gunshot from an automobile backfire using temperature alone. Achieving better results in this area requires the equivalent of combining multiple sensor data with the two-color radiometry data.
Achieving the very large dynamic range of operation required for military/security systems, combined with flexible operation, while providing new types of data for clutter rejection is a very difficult challenge. Simply extending current design approaches is not the answer. A new approach is needed.
In part, the inspiration for a sensor that can address the above needs can be drawn from biology and the human eye. The eye combines two types of sensors, rods and cones, each of which is optimized for different sensing regimes. The eye and neural pathways also allow for multiple modes of sensing, including fast reactions and “normal” scanning of a scene for more detail, which is significantly slower. Finally, the eye extends its dynamic range with the pupil, which provides fast dim/bright switching based on overall scene brightness.
Accordingly, it is an object of the present invention to provide a sensor that can sense very high-speed events, typically associated with weaponry. Examples include sensing projectiles such as bullets; tracking supersonic maneuvering missiles; sensing the launch flash from the ejection of a projectile from a weapon; and detecting gunshot muzzle flashes.
It is another object of the present invention to provide a sensor with the ability to support a high (large) dynamic range, i.e. the ability to simultaneously sense dim or low contrast objects along with bright or high contrast objects. Examples of dim/low contrast objects include cold objects in space, or concealed personnel against a warm earth background. Examples of bright or high contrast objects include gunfire or explosions.
It is a further object of the present invention to provide a sensor that can avoid saturation for very bright events, such as large explosions. If a sensor is saturated, it is temporarily blinded.
It is still a further object of the present invention to provide a sensor with the ability to produce a high-resolution image that shows detail.
It is still a further object of the present invention to provide data that can be used to improve methods to avoid false alarms from manmade signals or clutter, which typically limit sensor performance more than signal to noise for non-space applications.
The following patents may be relevant to the field of the invention:
U.S. Pat. No. 7,608,823 to Tennant, incorporated herein by reference, discloses a multimode focal plane array architecture with electrically isolated commons for independent sub-array biasing to accommodate large bias amplitude differences and different temporal bias profiles.
U.S. Pat. No. 7,075,079 to Wood, incorporated herein by reference, discloses a dual wavelength focal plane having a first array of infrared sensing pixel elements, and a second array of visible light pixel elements adapted to be selective to colors encountered while driving an automobile. The arrays are vertically stacked on a monolithic silicon substrate, and they are electrically coupled to a processor and display to integrate the infrared and color pixel elements into a view for a driver of the automobile.
U.S. Pat. No. 6,407,439 to Hier et al., incorporated by reference, discloses a plurality of photodetectors, each sensitive to different wavelengths, integrated on a common semiconductor substrate. The different photodetectors can be stacked over one another or placed laterally on the common substrate. Three of such photodetectors can form a pixel of an active matrix array for an image sensor. The different photodetectors in each pixel can be multiplexed electronically. The electronic circuits for activating the different photodetectors can be integrated on the same substrate.
U.S. Pat. No. 6,034,407 to Tennant, incorporated herein by reference, discloses multi-spectral planar photodiode pixels for simultaneously detecting multi-colors of infrared radiation. Each multi-spectral planar photodiode pixel includes a semiconductor substrate layer, a buffer layer of a first conductivity type material deposited on a semiconductor substrate layer, and a first color layer of the first conductivity type material deposited on the buffer layer. The multi-spectral planar photodiode pixel further includes a barrier layer of the first conductivity type material deposited on the first color layer, a second color layer of the first conductivity type material deposited on the barrier layer, and a cap layer of the first conductivity type material deposited on a the second color layer. A first diode comprising of a second conductivity type material is formed in the first color layer, and a second diode comprising a second conductivity type material is formed in the second layer.
U.S. Pat. No. 5,903,659 to Kilgore, incorporated herein by reference, discloses an adaptive method for removing fixed pattern noise from focal plane array (FPA) imagery (sensor images). A set of correction tenus is applied to a blurred version of the FPA image, and a filter is applied to the corrected, blurred image. Fixed pattern noise errors are then calculated using the filtered imagery, and employed to update the correction terms. The updated correction terms are then used for processing the next image. In one embodiment, the filter is an anti-median filter. In another embodiment, the filter is an anti-mean filter. These methods are commonly referred to as non-uniformity correction, or NUC.
U.S. Pat. No. 5,751,049 to Goodwin, incorporated herein by reference, discloses a two-colored infrared detector comprising elements having one or more diodes and a metal insulator semiconductor (MIS) device. The diodes are comprised of regions of semiconductor materials, which are operable to generate electron-hole pairs when struck by infrared radiation having first and second wavelengths. The capacitor includes a gate which is operable to generate a potential well in the first semiconductor region in conjunction with an insulator layer and collect charges generated by the first wavelength of infrared radiation. The layers of semiconductor material may be varied to enhance the performance of the resulting infrared device.
U.S. Pat. No. 5,583,338, to Goodwin, incorporated herein by reference, discloses a HgCdTe S-I-S (semiconductor-insulator-semiconductor) two color infrared detector wherein the semiconductor regions are group II-VI, with different compositions for the desired spectral regions. The device is operated as a simple integrating MIS device with respect to one semiconductor.
U.S. Pat. No. 5,559,336 to Kosai et al., incorporated herein by reference, discloses a radiation detector pixel unit cell with a n-p +LWIR photodiode that is vertically integrated with a p+n MWIR detector in a n-p+-n structure. Electrical contact is made separately to each of these layers in order to simultaneously detect both the LWIR and MWIR bands. The electrical contact is made via indium bump interconnections so that the detector unit cell can be subsequently hybridized with a topside mounted electronic readout integrated circuit. The n-p+-n structure in a given pixel of an array of radiation detector pixels is electrically isolated from all neighboring pixels by a trench that is etched into an underlying substrate.
U.S. Pat. No. 5,120,960 to Halvis, incorporated herein by reference, discloses an infrared (IR) imaging device with substantially identical top and bottom IR detector arrays. In separate embodiments, either a top or bottom surface of the top array is stacked onto the bottom array to confront a top surface of the bottom array. Individual detector elements and subarrays of the top array are aligned with corresponding detector elements and subarrays of the bottom array. The image readout circuits of both the top and bottom array are connected by wire bonding to readout control circuits formed in the peripheral region of the wafer in which the bottom array is formed. These IR detector arrays are formed on separate substrates. Furthermore, the two sub-arrays are aligned in a one-to-one pixel relationship, the purpose being to eliminate non-operational pixels from the combined dual array.